Object detection system

ABSTRACT

An airborne mine countermeasure system includes a processor coupled to a memory having stored therein software instructions that, when executed by the processor, cause the processor to perform a series of image processing operations. The operations include obtaining input image data from an external image sensor, and extracting a sequence of 2-D slices from the input image data. The operations also include performing a 3-D connected region analysis on the sequence of 2-D slices, and extracting 3-D invariant features in the image data. The operations further include performing coarse filtering, performing fine recognition and outputting an image processing result having an indication of the presence of any mines within the input image data.

Embodiments relate generally to mine detection and, more particularly,to airborne mine countermeasures.

Conventional approaches to underwater mine detection include using amatched filter to process 2-D image data. Conventional approaches can besubject to false positives, which can produce a larger subset of imagesthat may require human review. Moreover, the 2-D matched filter approachcan be sensitive to rotation and/or size changes, and also sensitive tointerfering objects.

Embodiments were conceived in light of the above problems andlimitations of some conventional systems, among other things.

One or more embodiments include an airborne mine countermeasure systemhaving a processor coupled to a memory having stored therein softwareinstructions that, when executed by the processor, cause the processorto perform a series of image processing operations. The operationsinclude obtaining input image data from an external image sensor, andextracting a sequence of 2-D slices from the input 3-D image data. Theoperations also include performing a 3-D connected region analysis onthe sequence of 2-D slices, and determining 3-D invariant features inthe image data. The operations further include performing coarsefiltering, performing fine recognition, and outputting an imageprocessing result having an indication of the presence of any mineswithin the input image data.

One or more embodiments can also include a computerized method fordetecting underwater mines. The method can include obtaining, at aprocessor, input image data from an external image sensor, andextracting, using the processor, a sequence of 2-D slices from the inputimage data. The method can further include performing a 3-D connectedregion analysis on the sequence of 2-D slices, and determining 3-Dinvariant features in the image data. The method can also includeperforming coarse filtering, performing fine recognition, and outputtingan image processing result having an indication of the presence of anymines within the input image data.

One or more embodiments can also include a nontransitory computerreadable medium having stored thereon software instructions that, whenexecuted by a processor, cause the processor to perform a series ofoperations. The operations can include obtaining, at a processor, inputimage data from an external image sensor, and extracting, using theprocessor, a sequence of 2-D slices from the input image data. Theoperations can further include performing a 3-D connected regionanalysis on the sequence of 2-D slices, determining 3-D invariantfeatures in the image data. The operations can also include performingcoarse filtering, performing fine recognition, and outputting a resulthaving an indication of the presence of any mine images within the inputimage data.

In one or more embodiments, the extracting can include using diffusionequations to generate a sequence of 2-D slices. In one or moreembodiments, performing the connected region analysis includesanalyzing, at the processor, volumetric pixel elements.

In one or more embodiments, performing the fine recognition includesapplying a metric including the Hausdorff metric. In one or moreembodiments, the extracting can include extracting different sizeobjects through a series of filtering operations.

In one or more embodiments, operations can further include performing acoarse filtering operation. Also, the nontransitory computer readablemedium can be configured to be executed by a processor onboard anaircraft.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of an exemplary airborne mine countermeasure systemin accordance with at least one embodiment.

FIG. 2 is a chart showing an exemplary airborne mine countermeasuremethod in accordance with at least one embodiment.

FIG. 3 is a diagram of an exemplary aircraft having an airborne minecountermeasure system in accordance with at least one embodiment.

FIG. 4 is a diagram of an exemplary 3-D invariant feature extraction andclassification in accordance with at least one embodiment.

DETAILED DESCRIPTION

FIG. 1 is a diagram of an exemplary airborne mine countermeasure system100. In particular, the system 100 includes an airborne minecountermeasures (AMCM) unit 102 coupled to a memory 104 and an interface106.

In operation, the AMCM module 102 receives input data 108 (e.g., imagedata) and processes the data according to a method for AMCM describedbelow. The image data can be obtained, for instance, from an imagesensor onboard an aircraft such as an airplane, unmanned aerial vehicle(UAV) or helicopter (see, e.g., FIG. 3, reference number 300). The AMCMmodule 102 can store a result of processing in the memory 104, and/orthe AMCM module 102 can transmit the result data to one or more externalsystems via the interface 106. The result can be provided as output data110 to another system, or sent to a display device or printer.

The output result can be an enhanced image showing potential underwatermines, which may not have been visible or as distinguishable in theinput image.

The interface 106 can be a specialized bus onboard an aircraft orvehicle (e.g., 1553, CAN bus, or the like), or a wired or wirelessnetwork interface.

FIG. 2 is a chart showing an exemplary airborne mine countermeasuremethod. Processing begins at 202 and continues to 204.

At 204, input data is received (e.g., image data). Processing continuesto 206.

At 206, a series of 2-D slices are extracted from the input data.Diffusion equations can be used to extract the series of 2-D slices, forexample, according to the following equations:

$\frac{\partial{u\left( {x,y,t} \right)}}{\partial t} = {\frac{\partial^{2}{u\left( {x,y,t} \right)}}{\partial x^{2}} + \frac{\partial^{2}{u\left( {x,y,t} \right)}}{\partial y^{2}}}$whereu(x, y, 0)=I(x, y) (the Original Image).With certain boundary condition, the solution is:

${u\left( {x,y,t} \right)} = {{{cI}\left( {x,y} \right)} \otimes {\exp\left( {- \frac{\left( {x^{2} + y^{2}} \right)}{2t}} \right)}}$and$\frac{\partial{u\left( {x,t} \right)}}{\partial t} = \frac{\partial^{2}{u\left( {x,t} \right)}}{\partial x^{2}}$whereu(x,0) (the Original Image).With certain boundary condition, the solution is:

${u\left( {x,t} \right)} = {{{cI}\left( {x,y} \right)} \otimes {\exp\left( {- \frac{\left( x^{2} \right)}{2t}} \right)}}$

The diffusion equation approach described above can permit a system toextract different size objects through a series of band-pass-likefiltering operations. The diffusion equation technique is described in“Repeatedly smoothing, discrete scale-space evolution, and dominantpoint detection,” by Bing C. Li, Pattern Recognition, Vol. 29, No. 6,1996, which is incorporated herein by reference. It will be appreciatedthat techniques other than diffusion equations can be used. Byextracting 2-D slices, image processing time can be improved. Processingcontinues to 208.

At 208, a 3-D connected region analysis is performed. The 3-D connectedregion analysis can be performed, for example, using voxels (orvolumetric pixel elements). The 3-D connected region analysis can helpreduce false positives. Processing continues to 210.

At 210, 3-D invariant features in the image data are extracted. 3-Dinvariant features are those features that do not substantially changewith respect to an aspect of an object such as size, orientation orother feature of that object in an image. The 3-D invariant features caninclude invariant moments, which are weighted averages (i.e., moments)of image pixel intensities, or a function of such moments. Invariantmoments are typically chosen to have some attractive property, such assize or orientation invariance. It is possible to calculate moments thatare invariant to translation, scale (or size) and/or rotation, forexample the Hu set of invariant moments is a common set of invariantmoments that can be used to extract invariant features. It will beappreciated that other invariant moments can be used.

A 3-D orientation invariant feature is a feature that can be extractedfrom an object in an image and is substantially the same regardless ofthe orientation of the object being imaged. A 3-D orientation invariantfeature could be used, for example, to identify an underwater mineregardless of the orientation of the mine when it was imaged. A sizeinvariant feature could be used to identify an object regardless of theapparent size of the object in the image. Object image sizes, even ofthe same object, can appear different for a number of reasons such asdistance from the object to the image sensor and the optics being used,for example. The 3-D invariant features can be size and/or orientationinvariant features. By converting the pixel data to representativeinvariant features, the specific location, size and orientation of anobject are replaced with invariant features that can be compared to areference catalog of invariant features of objects of interest (e.g.,different types of underwater mines). Thus, the invariant featurespermit a processor to recognize a 3-D object in the image dataregardless of its orientation, location or scale in the image.

The invariant features can include global or local invariants and can bebased on 3-D moment invariants. Processing continues to 212.

At 212, coarse filtering is performed. The coarse filtering can includecomputing a distance of invariant features in input image data toinvariant features in a feature catalog. The feature catalog can bebased on training data (see FIG. 4 and corresponding description belowfor more details). The distance can be a Euclidean distance, and thefiltering can including labeling objects in the input image dataaccording to the K nearest neighbors of those objects in the featurecatalog data. An output of the coarse filtering can include providing alist of the objects in the input image that most closely match one ormore objects in the feature catalog. The coarse filtering based oninvariant features can help speed up the image processing. Processingcontinues to 214.

At 214, fine object recognition is performed. The fine objectrecognition can be performed on the image data using a method with ametric, such as a Hausdorff metric. The fine object recognition canfocus on the list of objects supplied by the coarse filtering step (212)and can be used to help achieve a low false positive rate. An output ofthe fine filtering process can include a list of those objects that havebeen further determined to match the features of objects in the catalog(e.g., those objects that appear to match one or more underwater minetypes). Processing continues to 216.

At 216, a result of the mine countermeasure processing can be provided.The result can be displayed on a display device, sent to a printingdevice or transmitted to an external system via a wired or wirelessnetwork. Processing continues to 218, where processing ends.

It will be appreciated that 204-216 may be repeated in whole or in partin order to accomplish a contemplated airborne mine countermeasure task.

FIG. 3 is a diagram of an exemplary aircraft 300 having an airborne minecountermeasure system 100 in accordance with the present disclosure. Theairborne mine countermeasure system 100 is coupled to an image sensor302 that is adapted to take images of a body of water in order to detectan underwater mine 304. The aircraft 300 can be a fixed wing or rotarywing aircraft. Also, the aircraft 300 can be a manned aircraft orunmanned aerial vehicle (UAV).

In operation, the aircraft 300 uses an image sensor 302 to take imagesof a body of water in order to detect an underwater mine 304. Theairborne mine countermeasure system 100 is adapted to receive imagesfrom the image sensor 302 and process the images, as described above, inorder to detect an underwater mine 304. The processing can includeproviding an enhanced output display that indicates the presence ofobjects that are potentially underwater mines.

FIG. 4 is a diagram of an exemplary 3-D invariant feature extraction andclassification system 400. The system 400 includes a training sub-system402 having one or more training data sets 404, a 3-D invariant featureextraction module 406 and a 3-D invariant feature catalog 408.

The system 400 also includes a “live” data processing sub-system 410having live data 412, a 3-D invariant feature extraction module 414, aclassifier 416 and output result data 418. The live data being datacollected during operation of the system and not intended for primaryuse as training data. Live data is not limited to data being capturedand processed in real time, and can include operational image datacaptured and stored for later analysis.

In operation, a first phase includes extracting 3-D invariant featuresfrom the training data 404. The invariant features can include momentinvariant features. Also, the features can be size invariant and/ororientation invariants. The training data 404 can include images of oneor more known objects, such as underwater mines. The training data 404can include image data of a real object or data having a computergenerated image of an object. The process of extracting the 3-Dinvariant features can be supervised, semi-supervised and/orunsupervised.

The extracted 3-D invariant features can be cataloged and stored in adata store for later access. The cataloging process can be automatic,manual or a combination of the above.

The catalog of 3-D invariant features 408 can be updated to includeinvariant features of new objects (e.g., new types of underwater mines).

Once the catalog of 3-D invariant features 408 is generated, it can besupplied to a system (e.g., an airborne mine countermeasure system) foruse on live data.

The live data processing sub-system 410 takes in live image data 412captured by an image capture device and supplies the live image data 412as input to a 3-D invariant feature extraction module 414. The 3-Dinvariant feature extraction module 414 can be the same as (or differentfrom) the 3-D invariant feature extraction module 406 in the trainingsubsystem 402.

The 3-D invariant feature extraction module 414 extracts one or morefeatures from the live image data 412 and provides those features to theclassifier 416. The classifier 416 determines how close the extractedfeatures of the live image data match the features of one or moreobjects in the catalog. The classifier can use any suitable techniquesuch as a neural network, K nearest neighbors, or a supervised (e.g.,support vector machine or the like) or unsupervised machine learningtechnique. The classifier 416 can perform the coarse filtering functiondescribed above.

The output 418 of the classifier 416 can include a list of objects fromthe live image data and an indication of how close features of thoseobjects match features in the feature catalog 408.

The output results 418 can be used to determine if one or more of theobjects in the live image data 412 appear to be an object of interest(e.g., an underwater mine). Any objects of interest can be provided to amodule for fine object recognition to further determine with greateraccuracy or certainty that an object in an image is an object ofinterest.

It will be appreciated that the modules, processes, systems, andsections described above can be implemented in hardware, hardwareprogrammed by software, software instructions stored on a nontransitorycomputer readable medium or a combination of the above. A system forairborne mine countermeasures, for example, can include using aprocessor configured to execute a sequence of programmed instructionsstored on a nontransitory computer readable medium. For example, theprocessor can include, but not be limited to, a personal computer orworkstation or other such computing system that includes a processor,microprocessor, microcontroller device, or is comprised of control logicincluding integrated circuits such as, for example, an ApplicationSpecific Integrated Circuit (ASIC). The instructions can be compiledfrom source code instructions provided in accordance with a programminglanguage such as Java, C, C++, C#.net, assembly or the like. Theinstructions can also comprise code and data objects provided inaccordance with, for example, the Visual Basic™ language, or anotherstructured or object-oriented programming language. The sequence ofprogrammed instructions, or programmable logic device configurationsoftware, and data associated therewith can be stored in a nontransitorycomputer-readable medium such as a computer memory or storage devicewhich may be any suitable memory apparatus, such as, but not limited toROM, PROM, EEPROM, RAM, flash memory, disk drive and the like.

Furthermore, the modules, processes systems, and sections can beimplemented as a single processor or as a distributed processor.Further, it should be appreciated that the steps mentioned above may beperformed on a single or distributed processor (single and/ormulti-core, or cloud computing system). Also, the processes, systemcomponents, modules, and sub-modules described in the various figures ofand for embodiments above may be distributed across multiple computersor systems or may be co-located in a single processor or system.Exemplary structural embodiment alternatives suitable for implementingthe modules, sections, systems, means, or processes described herein areprovided below.

The modules, processors or systems described above can be implemented asa programmed general purpose computer, an electronic device programmedwith microcode, a hard-wired analog logic circuit, software stored on acomputer-readable medium or signal, an optical computing device, anetworked system of electronic and/or optical devices, a special purposecomputing device, an integrated circuit device, a semiconductor chip,and a software module or object stored on a computer-readable medium orsignal, for example.

Embodiments of the method and system (or their sub-components ormodules), may be implemented on a general-purpose computer, aspecial-purpose computer, a programmed microprocessor or microcontrollerand peripheral integrated circuit element, an ASIC or other integratedcircuit, a digital signal processor, a hardwired electronic or logiccircuit such as a discrete element circuit, a programmed logic circuitsuch as a PLD, PLA, FPGA, PAL, or the like. In general, any processorcapable of implementing the functions or steps described herein can beused to implement embodiments of the method, system, or a computerprogram product (software program stored on a nontransitory computerreadable medium).

Furthermore, embodiments of the disclosed method, system, and computerprogram product may be readily implemented, fully or partially, insoftware using, for example, object or object-oriented softwaredevelopment environments that provide portable source code that can beused on a variety of computer platforms. Alternatively, embodiments ofthe disclosed method, system, and computer program product can beimplemented partially or fully in hardware using, for example, standardlogic circuits or a VLSI design. Other hardware or software can be usedto implement embodiments depending on the speed and/or efficiencyrequirements of the systems, the particular function, and/or particularsoftware or hardware system, microprocessor, or microcomputer beingutilized. Embodiments of the method, system, and computer programproduct can be implemented in hardware and/or software using any knownor later developed systems or structures, devices and/or software bythose of ordinary skill in the applicable art from the functiondescription provided herein and with a general basic knowledge of thesoftware engineering and/or image processing arts.

Moreover, embodiments of the disclosed method, system, and computerprogram product can be implemented in software executed on a programmedgeneral purpose computer, a special purpose computer, a microprocessor,or the like.

It is, therefore, apparent that there is provided, in accordance withthe various embodiments disclosed herein, computer systems, methods andcomputer readable media for airborne mine countermeasures.

While the invention has been described in conjunction with a number ofembodiments, it is evident that many alternatives, modifications andvariations would be or are apparent to those of ordinary skill in theapplicable arts. Accordingly, Applicant intends to embrace all suchalternatives, modifications, equivalents and variations that are withinthe spirit and scope of the invention.

What is claimed is:
 1. An object detection system comprising: an imagesensor configured to obtain input image data of a body of water; aprocessor for executing modules; a memory containing the modules, theexecuted modules including: an image slice extraction module configuredto extract a sequence of 2-D slices from the input image data; a 3-Danalysis module configured to perform a 3-D connected region analysis onthe sequence of 2-D slices; an invariant feature computation moduleconfigured to compute image 3-D invariant features in the image data; animage filter module configured to perform coarse filtering based on theimage 3-D invariant features, the coarse filtering including: comparingimage 3-D invariant features in the image data to a data store oftrained 3-D invariant features associated with known objects, providinga list of found objects in the image data based on the data store, andfor each of the found objects, providing an indication of how closefeatures of the found object match features in the data store; and animage recognition module configured to perform fine recognition toidentify a known object from the data store within the input image data.2. The system of claim 1, further comprising: an object training moduleconfigured to extract training 3-D invariant features of known objectsin training data, and store the training 3-D invariant features as thetrained 3-D invariant features and the known objects in the data store.3. The system of claim 1, wherein performing the fine recognitionincludes applying a metric including a Hausdorff metric.
 4. The systemof claim 1, wherein the image slice extractor is further configured toapply diffusion equations to generate the sequence of 2-D slices.
 5. Thesystem of claim 1, wherein the 3-D invariant features include momentinvariants.
 6. The system of claim 1, wherein the image sensor islocated on an aircraft.
 7. The system of claim 6, wherein the aircraftis a helicopter, fixed wing aircraft, or unmanned aerial vehicle.
 8. Acomputerized method for detecting objects, comprising: obtaining, at aprocessor, input image data of a body of water from an image sensor;extracting, using the processor, a sequence of 2-D slices from the inputimage data; performing a 3-D connected region analysis, using theprocessor, on the sequence of 2-D slices; determining, using theprocessor, 3-D invariant features in the image data; performing, usingthe processor, coarse filtering based on the image 3-D invariantfeatures, the coarse filtering including: comparing image 3-D invariantfeatures in the image data to a data store of trained 3-D invariantfeatures associated with known objects, providing a list of foundobjects in the image data based on the data store, and for each of thefound objects, providing an indication of how close features of thefound object match features in the data store; and performing finerecognition to identify a known object from the data store within theinput image data.
 9. The method of claim 8, further comprising: trainingthe data store, the training comprising: extracting training 3-Dinvariant features of known objects in training data; and storing thetraining 3-D invariant features as the trained 3-D invariant featuresand the known objects in the data store.
 10. The method of claim 8,wherein performing the fine recognition includes applying a metricincluding a Hausdorff metric.
 11. The method of claim 8, wherein theextracting includes using diffusion equations to generate the sequenceof 2-D slices.
 12. The method of claim 8, wherein the 3-D invariantfeatures include moment invariants.
 13. The method of claim 8, whereinperforming the connected region analysis includes analyzing, at theprocessor, volumetric pixel elements.
 14. A nontransitory computerreadable medium having stored thereon instructions that, when executedby a processor, cause the processor to perform a series of operations todetect objects, the detecting including: obtaining, at a processor,input image data of a body of water from an image sensor; extracting,using the processor, a sequence of 2-D slices from the input image data;performing a 3-D connected region analysis, using the processor, on thesequence of 2-D slices; determining, using the processor, 3-D invariantfeatures of at least one object in the image data; performing, using theprocessor, coarse filtering based on the image 3-D invariant features,the coarse filtering including: comparing image 3-D invariant featuresof the at least one object to a data store of trained 3-D invariantfeatures associated with known objects and providing an indication ofhow close features of the at least one object match features of at leastone known object in the data store; and performing fine recognition todetermine if the at least one object in the image data corresponds to aknown object.
 15. The computer readable medium of claim 14, wherein thedetecting further includes: training the data store, the trainingcomprising: extracting training 3-D invariant features of known objectsin training data; and storing the training 3-D invariant features as thetrained 3-D invariant features and the known objects in the data store.16. The computer readable medium of claim 14, wherein performing thefine recognition includes applying a metric including a Hausdorffmetric.
 17. The computer readable medium of claim 14, wherein theextracting includes using diffusion equations to generate the sequenceof 2-D slices.
 18. The computer readable medium of claim 14, wherein the3-D invariant features include moment invariants.
 19. The computerreadable medium of claim 14, wherein performing the connected regionanalysis includes analyzing, at the processor, volumetric pixelelements.
 20. The nontransitory computer readable medium of claim 14,wherein the extracting includes extracting different size objectsthrough a series of filtering operations.